Skip to content

Model Serving integration - all in one#248

Closed
pkosiec wants to merge 14 commits intomainfrom
pkosiec/serving-3-playground-docs
Closed

Model Serving integration - all in one#248
pkosiec wants to merge 14 commits intomainfrom
pkosiec/serving-3-playground-docs

Conversation

@pkosiec
Copy link
Copy Markdown
Member

@pkosiec pkosiec commented Apr 3, 2026

Demo

model-serving-demo-compressed.mp4

See PRs:

PR1: #239
PR2: #240
PR3: #241

@pkosiec pkosiec force-pushed the pkosiec/serving-3-playground-docs branch 6 times, most recently from c2b8e5d to 04406bd Compare April 10, 2026 08:25
pkosiec added 14 commits April 10, 2026 15:25
Add Vite plugin that auto-generates TypeScript types from serving
endpoint OpenAPI schemas. Includes AST-based server file extraction
(@ast-grep/napi), schema-to-TypeScript conversion, and caching.
Also adds useServingInvoke and useServingStream React hooks in
appkit-ui with full type-safe registry support.

Signed-off-by: Pawel Kosiec <pawel.kosiec@databricks.com>
- Store requestKeys[] in serving cache instead of regex-parsing
  TypeScript type strings in schema-filter (fragile indentation dependency)
- Add overrideBody parameter to useServingStream's stream() to allow
  callers to pass fresh body without waiting for useMemo recomputation
- Lazy-init WorkspaceClient in type generator (skip when no endpoints resolve)

Signed-off-by: Pawel Kosiec <pawel.kosiec@databricks.com>
Chunks persisted after onComplete, causing the streaming bubble to
remain visible alongside the committed message (duplicate response).
Now chunks are cleared atomically with setStreaming(false) so React
batches all state updates in one render.

Signed-off-by: Pawel Kosiec <pawel.kosiec@databricks.com>
Clearing chunks in the hook's .then() handler caused a race with React
batching — chunks were empty before the component could commit them.
Let consumers decide when to clear via reset() instead.

Signed-off-by: Pawel Kosiec <pawel.kosiec@databricks.com>
Without a .catch(), if connectSSE rejects the promise is unhandled and
setStreaming(false) never fires, leaving the hook in a broken state.
This matches the pattern used by the genie chat hook.

Signed-off-by: Pawel Kosiec <pawel.kosiec@databricks.com>
The invoke callback is recreated whenever body changes (via useCallback
deps), which triggers the useEffect cleanup that aborts in-flight
requests. Adding overrideBody allows callers to use a stable body while
passing the real payload per-invocation, matching useServingStream.

Signed-off-by: Pawel Kosiec <pawel.kosiec@databricks.com>
…T_NAME in type generator

Signed-off-by: Pawel Kosiec <pawel.kosiec@databricks.com>
Integrate the Model Serving plugin into the dev-playground app with a
chat-style streaming demo page. Add serving plugin to app templates
and appkit init scaffolding. Include plugin documentation and
auto-generated API reference docs.

Signed-off-by: Pawel Kosiec <pawel.kosiec@databricks.com>
Build the full message array synchronously in handleSubmit and pass it
to stream() via the new overrideBody parameter, instead of relying on
useMemo recomputation via setTimeout.

Signed-off-by: Pawel Kosiec <pawel.kosiec@databricks.com>
Replace dual-source pattern (onComplete + streaming bubble) with a
useEffect that commits the assistant message on streaming→false
transition, then calls reset(). Eliminates both the duplicate
response and missing subsequent responses.

Signed-off-by: Pawel Kosiec <pawel.kosiec@databricks.com>
The Message interface includes an id field used as a React key. When
spreading messages into the API payload, this extra field was included,
which could cause the Databricks serving endpoint to reject subsequent
requests. The first message worked because the messages array was empty.

Signed-off-by: Pawel Kosiec <pawel.kosiec@databricks.com>
The template computed body via useMemo([messages, input]), causing the
invoke callback to be recreated on every state change. This triggered
the useEffect cleanup that aborted in-flight requests. Use a stable
empty body and pass the real messages via invoke(overrideBody).

Signed-off-by: Pawel Kosiec <pawel.kosiec@databricks.com>
…T_NAME in playground and docs

Also add execution context section documenting OBO-by-default behavior.

Signed-off-by: Pawel Kosiec <pawel.kosiec@databricks.com>
- Rename display string in dev-playground serving route
- Fix wrongly renamed DATABRICKS_SERVING_ENDPOINT_CLASSIFIER in docs

Signed-off-by: Pawel Kosiec <pawel.kosiec@databricks.com>
@pkosiec pkosiec force-pushed the pkosiec/serving-3-playground-docs branch from 04406bd to 8ac9638 Compare April 10, 2026 13:26
@pkosiec pkosiec closed this Apr 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant